The invention relates to creating a still image from multiple images, and more specifically to an image capture and processing system that enables image mosaicing.
Image mosaicing involves stitching together multiple separate sets of image information to create a composite still image. In particular, image mosaicing involves stitching together frames of digital image information that are captured by digital still cameras or digital video cameras. Image mosaicing is often utilized to generate a single panoramic image from a series of individual images.
The process of generating a high-quality image mosaic requires large digital storage and processing capacity. In order to provide the storage and processing capacity needed to create a high-quality image mosaic, a still or video digital camera can be connected to a personal computer equipped with an image mosaicing application. Image information captured by the digital camera is dumped directly from the digital camera to the personal computer and stitched together by the mosaicing application into an image mosaic. The image mosaic generated by the personal computer can be viewed through the personal computer, printed, edited, and/or permanently stored. A stand-alone digital camera could be used to capture a sequence of images and to generate a high-quality image mosaic, however, the processing and storage capacity that are required to generate a high-quality image mosaic would be cost prohibitive for typical mass-market hand-held digital cameras.
During the capture of image information that is to be utilized to create an image mosaic, it is desirable to view at least an approximate image mosaic in real-time in order to determine if the target scene has been entirely captured. However, real-time image mosaic building is an expensive operation, requiring more memory and processing power than is available in mass-market hand-held digital cameras.
In view of the limitations of known image mosaicing systems, what is needed is a system and method that enables image mosaicing to be accomplished through hand-held mass-market digital cameras. In addition, what is needed is a system and method that enables real-time display of image mosaics.
In an embodiment of the invention, an imaging system includes a camera for capturing image frames, a frame pre-screen unit for pre-screening the captured image frames, a frame store unit for storing the pre-screened frames, and a refined frame mosaicing unit for generating a refined mosaic from the pre-screened image frames that are stored in the frame store unit. Because frames are pre-screened by the frame pre-screen unit, the most useful frames for image mosaicing can be stored while less useful frames are discarded, thereby conserving limited storage capacity.
In an embodiment of the imaging system, the frame pre-screen unit also includes a coarse align unit for generating a coarse image mosaic from the pre-screened image frames, and a display for displaying the coarse image mosaic. In an embodiment, the coarse image mosaic is generated and displayed in real-time relative to the image frames that are captured by the camera.
In an embodiment of the imaging system, the display is located within the camera and in another embodiment, the frame pre-screen unit and the frame store unit are located within the camera.
In another embodiment of the invention, a hand-held camera includes a lens and detector array for capturing image frames, a frame pre-screen unit for pre-screening captured image frames, a display for displaying a coarse image mosaic of the pre-screened image frames, a frame store unit for storing the pre-screened image frames, and a data transfer system for transferring the pre-screened image frames from the frame store unit to a support system that generates a refined image mosaic from the pre-screened image frames.
In an embodiment of the hand-held camera, the frame pre-screen unit includes a coarse motion tracking unit for tracking the motion of the captured image frames and generating coarse motion vectors, a frame reject unit for rejecting a first image frame if the coarse motion vector of the first image frame does not exceed a motion threshold and for providing the first image frame to the frame store unit if the coarse motion vector does exceed the motion threshold, and a coarse align unit for generating the coarse image mosaic from the image frames having coarse motion vectors that exceed the motion threshold.
In another embodiment of the invention, an imaging system includes a camera and a support computer system connected to the camera by a real-time communications link. The camera includes a lens and detector array for capturing image frames. The support computer system includes a frame pre-screen unit for pre-screening the image frames that are captured by the camera and for outputting pre-screened image frames, a coarse align unit for generating a coarse image mosaic from the pre-screened image frames, a frame store unit for storing the pre-screened image frames that are output from the frame pre-screening unit, a refined image mosaicing unit for generating a refined image mosaic from the pre-screened image frames that are stored in the frame store unit, and a display for displaying the coarse image mosaic and the refined image mosaic.
In another embodiment of the invention, an image mosaicing method involves capturing image frames, making a coarse motion determination for a first image frame relative to a reference frame with the coarse motion determination being represented as a coarse motion vector, comparing the coarse motion vector for the first image frame to a motion threshold, rejecting the first image frame if the coarse motion vector for the first image frame does not exceed the motion threshold, and storing the first image frame if the coarse motion vector for the first image frame exceeds the motion threshold. An embodiment of the method includes an additional step of aligning image frames having coarse motion vectors that exceed the motion threshold to create a coarse image mosaic.
An advantage of the systems and method is that coarse image mosaics can be generated and displayed in real-time through a hand-held digital camera without large storage and processing capacity. Another advantage is that the pre-screened image frames can be utilized by a support computer system to generate refined image mosaics.